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Camera Apparatus with Saliencv Signal Generation 

The present invention relates to electronic camera apparatus comprising an electronic 
camera for developing an image signal of a viewed scene and further provided with 
means for developing an indication of saliency of the viewed scene. 

The electronic camera may be of the still or video variety, or any hybrid version. That 
is to say it may be adapted for example for providing one or more temporally spaced 
image frame signals in response to an input command (still camera), or for providing 
a continuous succession of image frame signals in response to an input command 
(video camera), or for providing a continuous sequence of image frame signals 
interspersed with still image signals (where the latter may have greater resolution if 
desired) (hybrid camera). 

Whereas formerly a camera user might be expected to take a great deal of care in 
composing a picture before recording it at the optimum time, particularly where 
photographic film was involved, the advent of electronic cameras with digital image 
recording, and more recently rapid increase in available practical memory size, has 
increased the ease with which a large number of still images, or a long video 
sequence, can be recorded electronically. This is tending to lead to situations where a 
camera is left recording over an extended period of time, with the intention of 
subsequently sifting through the stored images to select only the more interesting 
images or sequences for further processing. Such further processing might involve 
permanent storage, copying, reproduction, transmission and/or printing for example. 
In addition, the stored images may need to be organised and/or indexed for easy 
retrieval. 

Nevertheless, as the amount of stored image data increases, there comes a point at 
which the time required to analyse it all eventually outstrips the time and patience of 
the user. In such a case, the recorded images are then not used at all, so that 
potentially interesting images are lost, or all of the data must be viewed, so that the 
potentially interesting images are lost in a welter of uninteresting material. It can be 



seen that in either case the provision of too large a storage facility tends to be 
counterproductive. 

Our copending UK Patent Application No. GB 0118595.8 relates to automated 
camera apparatus comprising an electronic camera, the camera comprising a 
photoelectric sensor optically coupled to imaging optics for providing a video signal 
representative of an image viewed by the camera, and securing means for securing at 
least the imaging optics on the head of a wearer with its axis generally parallel to the 
normal forward line of sight of the wearer, the apparatus further comprising video 
signal processing means arranged to receive said video signal for judging the degree 
of interest shown by the wearer in a feature in the scene and for providing a related 
saliency signal, the video signal processing means including image analysis means for 
identifying the presence of features and their locations relative to a reference position 
in the frame within each of a series of frames in a time window of the video signal, 
and movement analysis means coupled to the output of said image analysis means for 
analysing the movement of at least one said feature over said series of frames to 
provide said saliency signal. 

This apparatus automatically derives a measure of saliency which is then used at the 
time or later for processing of the images or image signals from the camera. The 
saliency measure is derived from external cues such as way in which the gaze of the 
camera user alters. 

Another form of apparatus is described by Andrea Lockerd in "LAFCam - 
Leveraging Affective Feed back Camcorder", ACM CHI 2002 Conference on Human 
Factors in Computing Systems, Minneapolis, Minnesota, April 20-25, 2002. A video 
camera user wears a head mounted camera turned to view their own facial expression, 
a glove for sensing skin conductivity and a head mounted microphone for detecting 
laughter. The latter two can be used as saliency indicators when reviewing or editing 
the resulting video, and the facial expression can be presented as a picture within the 
video image for the same purpose. However, each of these measures is principally 
that of an involuntary action, not under the control of the user. 
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Other patent disclosures where a saliency or related measure is automatically 
determined include European Patent Application No. EP 1 109 132 (Eastman Kodak); 
US Patent No. 5,956,026 (Sharp); and International Patent Application No. WO 
00/79800 Telefonaktiebolaget LM Ericsson). A number of papers in this respect have 
5 also been published, examples being: - 

1 . "Summarizing Video Using a Shot Importance Measure and a Frame Packing 
Algorithm", by Shingo Uchihashi et al, in Proc ICASSP 1999, vol 6, pp 3041-3044. 

2. "Context Triggered Visual Episodic Memory Prosthesis 2, by Jyrki Hoisko, in 
Proc ISWC 2000, Atlanta GA. 

10 3. "Structuring Personal Activity Records Based on Attention- Analyzing Videos 
from Head Mounted Camera", by Yuichi Nakamura et al, in Proc ICPR 2000, 
Barcelona. 

4. "Video Retrieval and Relevance Feedback in the Context of a Post-Integration 
Model", by Rucyu Roy Wang et al, in Workshop on Multi-Media Signal Processing 
15 (MMSP2001), 3 rd to 5 th October 2001, Cannes, France. 

However, the most pertinent measure of saliency in many circumstances will be the 
amount of interest actually felt by the camera user, or the amount of importance 
actually attached to a picture by the user, and while external clues can give some 
indication of these factors, they cannot always be assessed accurately and 
2 0 automatically therefrom. 

Accordingly, the present invention provides camera apparatus comprising an 
electronic camera for producing an image signal and a physically or mechanically 
operable user control for receiving an input from a user and generating a saliency 
signal while the image signal is being produced. 

2 5 Many cameras incorporate both an on/off switch for energising the camera electronics 
prior to picture taking, and also a picture taking control. For example, a digital video 
camera may have an on/off switch which, inter aha, serves to enable the viewing 
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screen, together with a separate picture taking trigger which is operated for taking a 
video clip. A still photographic film camera, on the other hand, may include a single 
switch button having a first position for energising the electronics including camera 
settings such as focus, timing and aperture, and a further position at which the shutter 
is triggered. In both cases however, there is a clear distinction between the initial 
energisation step and the control of the picture taking. 

It may be considered that the mere operation of the camera button, trigger or other 
control for picture taking is an indication of saliency, particularly in situations where 
signal storage is limited. Commonly there is a resulting signal which is at least partly 
10 electrical, for example for operating the electronics of an electronic still or video 
camera to take and store image(s), or even for operating flash on an otherwise purely 
mechanical camera, and conventionally it will be binary in nature. 

However, as mentioned above, it would be desirable to be able to switch a camera on 
for taking pictures "continuously", for example all the time for video, or at regular 
15 periods for a still camera, regardless of the pictures it is taking, and to inspect or 
otherwise use the stored images at a later date. Under these circumstances, the fact of 
having the camera picture taking control actuated to provide a binary signal provides 
no indication whatsoever of the saliency of the stored images. 

Accordingly, in a first aspect, the apparatus of the present invention provides camera 
2 0 apparatus comprising an electronic camera for producing an image signal, a picture 
taking control causing the camera to take pictures, and additionally a physically or 
mechanically operable user control for receiving an input from a user and generating a 
saliency signal while the image signal is being produced. Thus the function of 
operating the camera to take pictures can be separated from the function of providing 
2 5 a saliency signal. In the first aspect of the invention, the saliency signal may be 
binary, multi-valued or continuous. 

In a second aspect, the present invention provides camera apparatus comprising an 
electronic camera for producing an image signal and a physically or mechanically 
operable user control for receiving an input from a user and generating a saliency 
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signal capable of exhibiting at least three values while the image signal is being 
produced. The user control may again be separate from a picture taking control of the 
camera. However in this case, where the saliency signal has more than two values, an 
alternative possibility is to use it both to control the camera for picture taking, 
5 commencing when a lower threshold value thereof is reached, and to employ the 
higher values thereof as indicative of saliency variations, in which case the user 
control may replace the normal on/off control. 

The saliency signal could be used in real time, e.g. to control operation of the camera, 
as does a signal from a picture taking control. However, unlike a conventional picture 

1 0 taking control signal, it may additionally or alternatively be stored for later use in 
image signal processing. Accordingly, in a third aspect, the present invention provides 
camera apparatus comprising an electronic camera for producing an image signal and 
a physically or mechanically operable user control for receiving an input from a user 
and generating a saliency signal while the image signal is being produced, wherein the 

15 apparatus also comprises means for storing the saliency signal together with the image 
signal. 

European Patent Application No. EP 0 860 980 describes a digital camera with the 
capability of allowing the user to examine pictures already stored, to decide various 
parameters, such as the number quality and size of any desired prints, which 
2 0 parameters are then stored for later use. However, as has already been discussed 
above, this means that the camera user has to review each of the images after they 
have been taken. 

By contrast, the present invention can permit an immediate attribution of saliency 
while the pictures are being taken, thereby reducing the amount of time and effort 

2 5 needed by the user to obtain the most desirable pictures. In some respects, this may be 

viewed as a development of the conventional camera picture button, where the user 
operates the button once the required picture is in view, i.e. once it has been decided 
that the picture has sufficient saliency. Where the saliency signal is stored, this 
enables relatively easy selection of the requisite picture at a later time from a larger 

3 0 number of images. Furthermore, there is relatively little additional effort in obtaining 



a multi-valued saliency signal during picture taking, so that refinement of a picture 
selection process (for example) is possible with little or no additional work once the 
picture and saliency signals have been recorded. 

In addition, as will be explained later, an immediate attribution of saliency can be 
5 useful in other ways, such as managing the memory, and controlling the transmission 
and/or compression of image signals, which features are not available on conventional 
camera apparatus. 

The user control is either a part of the camera itself, e.g. a part of the body or 
physically attached coupled or mounted thereto, or a remote user control capable of 
1 0 communicating with the camera or another part of the camera apparatus. 

, The user control may comprise a control member which is physically movable, as in 
the depression of a button, the operation of a trigger, or the movement of a slider or 
rotary knob with or without a self-return mechanism. Alternatively user control may 
include a control member which comprises a pressure or force sensing transducer 
15 such as a piezo-electric sensor. 

Where the control member is different from the picture taking control, as occurs in 
the first aspect of the invention, and as may also occur in the second and third aspects, 
it may be physically coupled to the camera body, or it may be remote from the camera 
body. Alternatively, in the second and third aspects of the invention, the control 
2 0 member could replace or be combined with the normal activation or picture taking 
control, so that a minimum level of the saliency signal is required to commence 
picture taking. 

Where the control member is coupled to the body it may be directly mounted thereon, 
for example the slider of a sliding potentiometer, or knob of a rotary potentiometer; or 
2 5 it may include a physically operable portion which is displaced from the body, as for 
example a remote mechanical release cable attached to sensor such as a physical 
movement or pressure/force sensor mounted to the camera body; or it may include an 
attachment separate from the body, for example an electrical trigger attachment 
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operable by movement or force/pressure, which is physically coupled by wire to the 
electronics of the camera within the body. 

Where the control member is remote from the camera body, it may include a 
movement sensor or a pressure/force sensor, the output of which is coupled by a 
5 transmitter to an appropriate receiver on the camera or another part of the camera 
apparatus by any well known mechanism, such as radio, ultrasound, or infra-red 
transmission. 

The use of a remote user control can be advantageous, particularly where the user is 
able merely to turn the camera on and leave it on. In such a case, if the camera is 

10 suitably mounted either on the user's body (for example in a pair of glasses worn on 
the head of the user) or at a location remote from the user, the user's hands are freed 
for most of the time. The remote user control may then be held by the user (for 
example a control in the form of a key fob), or worn on the user's body or clothing. 
Exemplary mountings would be as a ring, for example a ring to be worn on the index 

15 finger for operation of a slider moving round the ring by the thumb or incorporating a 
pressure sensor receiving pressure from the thumb, or a pin-on mounting to be worn 
on the user's tie, jacket, blouse, etc. 

The user control may initially provide an electrical output with a continuous range of 
values, e.g. from a linear or rotary potentiometer (or equivalent such as an 

2 0 arrangement including a Hall effect sensor), or a force/pressure sensor. If desired, this 
can be converted to a binary or discrete multi-value output, using conventional 
techniques such as thresholding. Alternatively, the user control may be arranged to 
provide a binary or discrete multi-value output directly, even from a continuously 
variable movement or pressure/force, for example a conventional on/off switch, or a 

2 5 switch with multiple contacts . 

The saliency signal may likewise have a continuous range of values, or be binary, or 
have multiple discrete values. It may be derived directly from the output of the user 
control, or from such output after thresholding. However, it can be derived in other 
ways. For example, where the user control provides a binary output, the frequency of 
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such output could be converted to a corresponding continuous or discrete multi-value 
saliency signal. 

Another option is to employ the binary output to increment or decrement a discrete 
multi-value saliency signal. This may require a further control element to instruct 
whether to increment or decrement, or a centre-biased three-position switch could be 
provided, for example, the positions either side of the spring biased centre rest 
position being for continuously or step-wise incrementing and decrementing 
respectively. Where the prevailing level of the saliency signal is not evident from the 
physical setting of the control, a small display of the value would be desirable. 

The camera apparatus may include at least one other means for developing a saliency 
signal. This could be a means for developing saliency signal as described in our 
copending UK Patent Application No. GB 01 1 8595.8 mentioned previously, or even a 
laughter detector or a glove for measuring skin conductivity. Alternatively or 
additionally, it could be one or more other physically or mechanically operable user 
controls operating in a manner similar to that described above in relation to the first 
user control, and each of the user controls could be associated respectively with a 
different type of expressed interest, for example interest in people, buildings, or 
events. 

The different individual saliency signals could be combined in real time, or used or 
recorded separately. Combination of saliency signals could be effected algebraically 
or logically to give a composite saliency signal; effective combination could also be 
effected by ascertaining (for example via a look-up table) into what portion of a 
multidimensional saliency space the saliency signals are jointly located, and 
developing a composite saliency signal. 

In use, the saliency signal(s) can be controlled by the camera user to indicate the 
amount of interest in the viewed scene as the camera is producing the image signal 
therefrom, and/or to indicate at which times it is desired preferentially to retain 
images of the viewed scene. In part, the user's actions may be influenced by the 
intended use of the saliency signal. In a variation, the saliency signal could be pre-set 
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by the user just prior to picture taking, e.g. where the user is already aware of the 
likely importance of, or interest in, the pictures. 

Where the saliency signal is recorded together with the image signal, later assessment 
or other processing of the stored images is facilitated. The recorded saliency signal 
5 may include a link or pointer to the associated image signal, or it may form part of the 
associated image signal, e.g. metadata. 

Thus, for example the stored saliency figure may be employed: - 

1. During replay to select for viewing only those portions of the stored image 
signal which have a saliency level above a threshold level (here and elsewhere 

10 "portions of the image signal" encompasses clips from a video signal, or (selected) 
still images, or both); 

2. During replay to select for accelerated viewing those portions of the stored 
image signal which have a saliency level below a threshold level; 

3. During re-recording, in the same memory or another memory, to select only 
15 those portions of the stored image signal which have a saliency level above a 

threshold level, and to reject the remainder of the signal. This could be done 
automatically during use of the camera to free up portions of the memory containing 
unwanted signals, and so extend the time during which the camera may be operated. 

4. During re-recording to selectively compress the image signal according to the 
2 0 saliency level. This may be done in conjunction with option 3 where there are more 

than two saliency levels. Highly salient signals may not be compressed at all, or only 
lightly compressed, whereas signals with little or no saliency can be rejected, or 
highly compressed. 

To be effective, and particularly when the signal is a video signal, options 3 and 4 
2 5 may need to take account of the current saliency level under consideration and the 
related historical saliency, i.e. its context. For example, it is probable that a sequence 
of closely spaced medium saliency periods relates to an action sequence, and so the 
whole portion of video containing such a sequence should be retained or treated as 

9 



being more salient than a single isolated period with the same recorded saliency level. 
This could be done by using some form of integration of the saliency level, with a 
relatively slow decay, or by some other form of smoothing. It may be necessary to 
use some form of delay so as to be able to analyse a previous portion of signal once an 
above-threshold integrated saliency level has been observed, in order to cover the 
whole of a salient sequence. 

Alternatively, or additionally, the saliency signal may be employed in real time to 
control the camera itself. For example, if the camera is set to take time-lapse still 
pictures, the frequency of picture taking could be controlled by the saliency level. In 
10 another example, if the camera can take both video and also higher resolution still 
pictures, recordal of still pictures in addition to video may be triggered only when the 
saliency is above a threshold level. 

The saliency signal could also be used in real time to control the amount of 
compression of the image signal to be recorded or transmitted, and/or to select which 

15 parts of the image signal are recorded or transmitted. In this type of operation, it could 
be appropriate to employ a buffer for the image signal prior to recordal or 
transmission, in order that some of the previous image signal may also be selected 
when an above threshold saliency level is detected, i.e. again taking account of the 
historical context of the saliency level. The buffer size could be adaptively changed 

2 0 depending on the current saliency level, so that during periods of relatively high 
saliency a larger portion of the image signal is stored in the buffer for potential 
recordal or transmission. 

The level of saliency required for different actions may be different. For example, 
where a camera is both transmitting and recording an image signal, the saliency level 
2 5 necessary for transmission may be greater than the level required for storage at the 
camera. 

Further features and advantages of the invention will become apparent upon a perusal 
of the appended claims, to which the reader is referred, and also upon a reading of the 
following more detailed description of embodiment of the invention, in which: 
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Figure 1 shows in outline schematic form an electronic video camera for use in an 
embodiment of apparatus according to all three aspects of the invention; 

Figure 2 shows in outline schematic form a physically operable remote saliency 
control for use in the camera of Figure 1; 

5 Figure 3 shows in outline schematic form the circuitry of the camera of Figure 1 ; 

Figure 4 shows in schematic perspective view an electronic still camera for use in an 
embodiment of apparatus according to all three aspects of the invention; and 

Figure 5 shows in outline schematic form the circuitry of the camera of Figure 4; 

The video camera 1 of Figure 1 comprises a main body 2 at the front of which is 
1 0 mounted the lens 3 and to which, nearer the rear end, is attached a grip 4 including a 
manual operating (picture taking) button or trigger 14. A three position rotary or 
slider switch 5 is mounted on the body 2, the first position 5 1 being "OFF", the second 
position 5 2 providing a manual control signal 6, and the third position 5 3 providing a 
continuous control signal 7, see Figure 3. Also provided on the camera body 2 is a 
15 remote signal sensor 8. A viewing screen 7 is arranged on the body so that in use it 
extends sideways from the camera body. 

Figure 2 shows a remote user control 9 suitable for use with the camera of Figure 1 
when the switch 5 is in its third position. As shown, it comprises a ring 10 intended 
to be worn on the index finger of the user. A potentiometer with a knob 1 1 slidable 

2 0 about the ring circumference within a range indicated by the dashed lines is mounted 
on the ring for operation by the user's thumb, and the potentiometer is coupled to a 
transducing arrangement 12 which reads the continuously variable potentiometer 
setting, converts it to a digital value with a desired degree of resolution, and transmits 
the digital value as an appropriately coded signal 13. By way of example only, there 

25 may be 4 digital values within the potentiometer movement range, and the 
transmission is by infra-red, ultrasonic or radio waves. Preferably, the potentiometer 
includes gentle click stops for the knob 11 corresponding to each of the 4 digital 
values. 
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It will be appreciated that the control 9 could take an alternative form to ring 10, for 
example a tie pin, badge or brooch, and that the slidable mechanism could be replaced 
for example by a rotary one. Also a transducer other than a potentiometer could be 
used, for example a multi-position switch. In another embodiment, the transducer is a 
piezoelectric pressure sensor. 

The circuitry within the camera is generally shown in Figure 3, where the first 
position 5i of the switch 5 merely turns everything off. Signals 6 and 7 are coupled 
to an OR gate 15 to produce a camera on signal 17, so that in either the second or 
third position of switch 5 the image sensor turns the camera electronics on, including 
an imaging sensor 16 receiving light from the lens 3 and the viewing screen 7. It will 
be appreciated that the signal 17 is coupled to other circuitry of the camera, including 
some of the other components shown in Figure 3, although for simplicity these 
comiections are omitted. The screen 7 is coupled to receive the image signal 18 from 
the image sensor 16 so that the viewing screen can show at that stage what the camera 
is seeing. 

The camera circuitry also comprises a memory 19 which is coupled to receive the 
image signal 18, but which is arranged only to store the signal when so instructed by 
an image control circuit 20. 

With the switch in the second position, the user can depress the trigger or button 14 in 
the grip 4 to transmit the manual control signal 6 to the memory control circuit 20 so 
that the memory is activated to store the image signal 18 in the normal way. As 
shown, switch 14 is a simple mechanical single pole switch, but any equivalent 
mechanical or electrical arrangement could be used. A further maximum saliency 
generator 21 is arranged to respond to the presence of an output from switch 21, and 
generates a maximum saliency signal 22 which is fed to a comparator 23. 
Comparator 23 receives a second input 24, and provides a saliency output 25 equal to 
the value of the greater of the two inputs 22, 24, the output 25 also being fed to the 
memory 19 to be associated with the image signal being stored. For example, both 
signals 18 and 25 may be provided with the same time stamp. In an alternative form 
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of apparatus, the signal 25 may be inserted into the existing image signal 18 in known 
manner prior to storage in memory 19. 

When the switch is in the third position 5 3 the continuous control signal 7 is used to 
enable the remote receiving sensor 8 for receiving and decoding a signal from the 
remote control 9 to provide a multi-level digital saliency signal 24 to the comparator 
23. Signal 7 is also coupled to the memory control circuit 20 to instruct the memory 
continuously to record the image signal 18 together with the saliency output 25. 

Thus, when the switch 5 is in the second position 5 2 the camera is operative to store 
image signals 18 and an associated maximum saliency output 25 only when the button 
14 is operated. When the button is not operated, no storage of image or saliency 
signals occurs, and the signal 22 assumes a zero saliency value. 

When the switch 5 is in the third position 5 3 the camera is operative continuously to 
store image and saliency signals as received from the image sensor 16 and the 
receiver 8 respectively. During this operation, should the button 14 be depressed, the 
maximum saliency signal 22 will predominate in the comparator and will be stored in 
the memory with the image signal. It will be understood that the circuit 21 is 
arranged to provide a maximum saliency signal, e.g. equivalent to or greater than the 
maximum signal received from the transmitter 9, since it is likely that operation of the 
button 14 has occurred because the user recognises a scene of high salience. 

The memory control circuit is also arranged periodically to check the amount of used 
memory. When this reaches a threshold figure, it co-operates with the memory 19 by 
rejecting stored images with associated low saliency levels (below a predesignated 
threshold level) and rearranging the remaining stored image signals to maximise 
memory availability. In an alternative embodiment, the control circuit 20 also 
includes one or more image coders, and when the memory is becoming full it begins 
to code the stored images with a degree of reproducibility determined by the 
associated saliency level, so that low saliency images are stored with minimal 
memory and correspondingly low resolution in the reproduced image, whereas high 
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saliency signals are coded and stored with greater memory occupancy so as to be 
capable of being more faithfully reproduced. 

In a variation of the embodiment of Figures 1 to 3 , the saliency signal is a binary 
signal only. In such a case, the potentiometer of Figure 2 may be replaced by a simple 
5 make switch for actuation by pressure from the user's thumb, the output of the switch 
enabling the transmitter to output a signal. At the camera, the comparator 23 may be 
replaced by a simple OR circuit. The image signals are therefore stored in memory 19 
together with a two-level saliency signal, and at a later stage the higher value of the 
latter signal may be used to select the associated images so to skim through the stored 
1 0 images and pick out those considered at the time to be more interesting. 

Figures 4 and 5 relate to an electronic still camera in which the rate of image 
acquisition and storage is controlled by a saliency signal. The saliency signal may or 
may not be stored together with the image signal. In these Figures the same numerals 
will be used for electrical components common to the embodiment of Figures 1 to 3. 

15 Figure 4 shows a rear view of the electronic still camera 30 having a body 31 on the 
rear surface of which is an electronic viewfinder 7 and the vertical slider of a three 
position switch 5. On the top of the body is a picture taking button 14, and on the 
right hand side of the body is a remote receiving transducer 8 for receiving a multi- 
value saliency signal from a remote transmitter such as the one shown in Figure 2. 

2 0 Much of the camera circuitry shown in Figure 5 is similar to that of Figure 3. 
However, a variable compression circuit is arranged between the image sensor 16 and 
the memory 19 and is controlled by the output of comparator 23. As shown, the 
output of comparator 23 is not coupled to the memory 19, although an optional 
connection is shown in dashed lines. Thus the saliency signal is not necessarily stored 

2 5 in memory 19 but it does determine the compression of the stored images. It is also 
used to determine the rate of image storage 

The first position 5 j of the switch tunas all of the circuitry off. 
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The second position 52 turns the camera electronics on and enables the viewer 7. The 
button 14 is coupled to a first input of a memory control circuit 20' so that when the 
button is depressed the control circuit enable storage of a single still image in memory 
19. Since the maximum saliency signal generator is operative, the compression 
circuit 26 is controlled to provide relatively little or no compression of the image 
signal. 

When the switch is in the third position 5 3 the camera circuitry including the receiving 
transducer 8 is also energised, and the received saliency signal 24 is passed to a 
second input of the memory control circuit 20', which enables storage of images in 
memory 19 according to the value of signal 24. In particular, when no signal 24 is 
present, or it has a below threshold value, no images are stored (or images are stored 
at a lowest rate), but above a threshold value image storage commences with a 
frequency which increases with the value of signal 24. 

In a variation, the saliency signal has only two levels, and the rate of image storage 
correspondingly varies between a lower (zero or finite) rate, and a higher rate. 

When the switch is in the third position 53 the camera can be operated for single 
picture taking using the button 14, at which time it is assumed that the saliency is at a 
maximum, or by use of the remote transmitter 9. 

It will be appreciated that the remote receiver 8 of either embodiment could be 
replaced by a user control on the camera itself providing the saliency signal directly to 
the camera electronics, or that the wireless coupling could be replaced by a hard wired 
connection. 

Furthermore, although in the embodiments both the video and saliency signals are 
directed to processing circuitry and/or a memory within the camera itself, these 
signals may be transmitted by the camera for remote processing and/or storage if 
desired. Indeed, it would be possible to distribute the processing circuitry and/or 
memory between the camera and a remote location (for example storing the video 
signal in the camera and storing the saliency signal remotely), provided there is a 
means of retaining an indication of synchronicity between the signals, for example by 
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referencing both signals to the camera time stamp, which would thus need to be 
transmitted the remote location. 



Where the image signal is buffered for a significant length of time, another use of the 
saliency signal would be to retrieve some of the past image signal, for example to an 
5 extent determined by the magnitude of the saliency signal. Thus the camera could be 
set to have the buffer active; when an event occurs which the user considers to be 
important or interesting to a certain degree, the user control is operated to a 
corresponding extent, and a corresponding length of the image signal stored in the 
buffer (for the maximum saliency level this could be all of the buffered signal) is then 
10 passed to a more permanent store. In this way the user could ensure that sudden 
events are not inadvertently missed. 
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CLAIMS 



1. Camera apparatus comprising an electronic camera for producing an image 
signal, a picture taking control causing the camera to take pictures, and additionally a 
physically or mechanically operable user control for receiving an input from a user 

5 and generating a saliency signal while the image signal is being produced. 

2. Camera apparatus comprising an electronic camera for producing an image 
signal and a physically or mechanically operable user control for receiving an input 
from a user and generating a saliency signal capable of exhibiting at least three values 
while the image signal is being produced. 

10 3. Apparatus according to claim 2 wherein the user control is separate from a 
picture taking control of the camera. 

4. Apparatus according to any preceding claim and including control means 
responsive to the saliency signal for controlling operation of the camera apparatus. 

5. Camera apparatus according to claim 4 and including compression circuitry 
15 for compressing the said image signals, the compression circuitry being arranged to 

receive the related saliency signals and to compress the image signals to an extent 
determined by the related saliency signals. 

6. Apparatus according to any preceding claim wherein the apparatus also 
comprises memory means for storing the saliency signal together with the image 

2 0 signal. 

7. Camera apparatus comprising an electronic camera for producing an image 
signal and a physically or mechanically operable user control for receiving an input 
from a user and generating a saliency signal while the image signal is being produced, 
wherein the apparatus also comprises memory means for storing the saliency signal 

2 5 together with the image signal. 
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8. Camera apparatus according to claim 6 or claim 7 and including memory 
management circuitry for selectively retaining images with higher saliency levels in 
said memory in preference to images with lower saliency levels. 

9. Camera apparatus according to any one of claims 6 to 8 and including image 
5 selection circuitry for receiving the saliency and image signals and for selectively 

passing ones of said image signals as determined by said saliency signals. 

10. Camera apparatus according to any one of claims 6 to 9 and including a buffer 
for receiving said image signal, the buffer length being controlled by the saliency 
signal. 

10 11. Camera apparatus according to any preceding claim and comprising at least 
one further physically or mechanically operable user control for generating a 
corresponding related saliency signal. 

12. Camera apparatus according to any preceding claim and comprising saliency 
circuitry for generating an image related saliency signal in response to the image 

15 signal. 

13. Camera apparatus according to claim 11 or claim 12 and comprising saliency 
circuitry for combining the saliency signals to provide a complex saliency signal. 

14. Camera apparatus according to any preceding claim and including circuitry for 
incorporating the related saliency signal in each said image signal. 

2 0 15. Camera apparatus according to any preceding claim wherein the user control 
forms part of the body of the camera itself or is physically attached thereto. 

16. Camera apparatus according to any one of claims 1 to 14 wherein the user 
control is a remote control capable of communicating with the camera. 

17. Camera apparatus according to any preceding claim wherein the user control 
2 5 comprises a physically movable control member and means for sensing the movement 

of the control member. 
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18. Camera apparatus according to any one of claims 1 to 16 wherein the user 
control comprises a pressure or force sensing transducer. 

19. Camera apparatus according to any preceding claim wherein the saliency 
signal has a plurality of values, or is continuously variable. 

20. Camera apparatus according to any preceding claim wherein the user control 
also provides the normal picture taking control on the camera. 

21 . Camera apparatus according to any one of claims 1 to 18 wherein the saliency 
signal is binary. 

22. Camera apparatus comprising an electronic camera for producing an image 
signal and a physically or mechanically operable user control for receiving an input 
from a user and generating a saliency signal while the image signal is being produced. 
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ABSTRACT 



Camera apparatus comprises an electronic still, video or hybrid camera 1 for 
producing an image signal and a physically or mechanically operable user control for 
receiving an input from a user and generating a binary, continuous or multi-level 
saliency signal while the image signal is being produced. The user control may be 
common with the picture taking control 14 or different (for the figure shown, it is a 
remote control communicating with the camera via a sensor 8). The saliency signal 
may be stored with the image signal for later vise, as in editing; or it may control 
processing of the camera signal in real time; or it may control the camera itself, for 
example by controlling the frequency of still picture taking, or by optimising image 
storage in memory. The user control may be mounted on the camera or remote 
therefrom and may include a movement or pressure sensor. 



(Figure 1) 




FIG. 3 



